Endoscopic surgery image processing apparatus, image processing method, and program

ABSTRACT

A medical image processing apparatus including a controller including circuitry configured to determine a position of a distal end of an important object within a medical image, estimate a region of interest within the medical image adjacent to a region including the important object based on the position of the distal end of the important object, and control display of the region of interest.

TECHNICAL FIELD

The present technology relates to an image processing apparatus, animage processing method, and a program, particularly, to an imageprocessing apparatus, an image processing method, and a program allowedto display a surgery region desired by a practitioner without an effortof the practitioner.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2014-115769 filed Jun. 4, 2014, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND ART

An endoscopic surgery has been used in which an endoscope is insertedinto a body, a region (surgery region) being a surgical target in thebody is captured as an observed portion to display the captured regionon a screen by using the endoscope, and treatment is performed on thesurgery region while viewing the screen. In the endoscopic surgery,desired signal processing is performed on an image signal of theobserved portion which has an optical image and is obtained in theendoscope by applying illumination light to the observed portion from alight source device and an image of the observed portion having anoptical image is displayed on a screen.

In such an endoscopic surgery, it is necessary that a range or aposition of an observed portion to be displayed on the screen isappropriately adjusted depending on circumstances in order for apractitioner performing surgery to ensure an optimal view field(surgical field) for a surgery region.

However, typically, the practitioner holds a surgical instrument forperforming surgery with both hands. Accordingly, it is difficult for thepractitioner to operate a work of such the screen adjustment rapidly forhimself. The practitioner operating an adjustment mechanism and the likefor himself for screen adjustment is not preferable in view of ensuringa degree of cleanness of a surgery region, medical equipment, anoperating room, and the like.

Thus, in general, an instruction of the practitioner is given to anassistant called a scopist or the like and the assistant operates theadjustment mechanism in accordance with the instruction from thepractitioner to perform such the screen adjustment.

However, in the method in which the assistant intervenes, theinstruction of the practitioner may be transferred inaccurately and thusrapid screen adjustment desired by the practitioner may be difficult.

As a method of adjusting a screen without an assistant intervening, forexample, PTL 1 discloses that focus control is performed on an area inwhich brightness or contrast does not change for a predetermined period.

CITATION LIST Patent Literature

PTL 1: Japanese Unexamined Patent Application Publication No.2011-139760

SUMMARY OF INVENTION Technical Problem

However, according to PTL 1, an area at which brightness or contrastdoes not change for a predetermined period may or may not be an areawhich a practitioner wishes to match with a focus and thus an incorrectfocus may be obtained.

The present technology is for performing estimation of region ofinterest desired by a practitioner without an effort of the practitionerin view of such circumstances.

Solution to Problem

According to an embodiment of the present disclosure, there is provideda medical image processing apparatus including a controller includingcircuitry configured to determine a position of a distal end of animportant object within a medical image, estimate a region of interestwithin the medical image adjacent to a region including the importantobject based on the position of the distal end of the important object,and control display of the region of interest.

According to another embodiment of the present disclosure, there isprovided a method for processing a medical image by a medical imageprocessing apparatus including a controller including circuitry. Themethod includes the steps of determining, using the circuitry, aposition of a distal end of an important object within the medicalimage, estimating, using the circuitry, a region of interest within themedical image adjacent to a region including the important object basedon the position of the distal end of the important object, andcontrolling, using the circuitry, display of the region of interest.

According to another embodiment of the present disclosure, there isprovided a medical image processing system including a medical imagingdevice that obtains a medical image, a display device having a displayarea, and a controller including circuitry configured to determine aposition of a distal end of an important object within the medical imageobtained by the medical imaging device, estimate a region of interestwithin the medical image adjacent to a region including the importantobject based on the position of the distal end of the important object,and control display of the region of interest in the display area of thedisplay device.

According to another embodiment of the present disclosure, there isprovided a medical image processing apparatus including a controllerincluding circuitry configured to determine a position of an importantarea within a medical image, estimate a region of interest within themedical image adjacent to a region including the important area based onthe position of the important area, and control display of the region ofinterest.

Advantageous Effects of Invention

According to the embodiments of the present technology, it is possibleto perform estimation of region of interest desired by the practitioner.

The effect described herein is not necessarily limited thereto and mayinclude any effect described in the present technology.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of anembodiment of an endoscope system according to the present technology.

FIG. 2 is a diagram illustrating a usage state of the endoscope system.

FIG. 3 is a diagram illustrating a calculation method of a depthposition.

FIG. 4 is a diagram illustrating the calculation method of the depthposition.

FIG. 5A is a diagram illustrating detection of a position of forceps.

FIG. 5B is a diagram illustrating detection of the position of forceps.

FIG. 5C is a diagram illustrating detection of the position of forceps.

FIG. 5D is a diagram illustrating detection of the position of forceps.

FIG. 6 is a diagram illustrating detection of a position of a remarkablepoint.

FIG. 7 is a diagram illustrating an example of a superposition image tobe displayed on a display.

FIG. 8 is a flowchart illustrating focus control.

FIG. 9 is a flowchart illustrating a forceps position detecting processin detail.

FIG. 10 is a flowchart illustrating a remarkable point estimatingprocess in detail.

FIG. 11 is a flowchart illustrating focus control.

FIG. 12 is a block diagram illustrating a configuration example of anembodiment of a computer according to the present technology.

DESCRIPTION OF EMBODIMENTS

Hereinafter, a configuration (below referred to as an embodiment) forimplementing the present technology will be described. The descriptionwill be made in the following order.

1. Configuration Example of Endoscope System

2. Flow of Focusing Process

3. The Other Flow of Focusing Process

4. Regarding Recording Medium

<Configuration Example of Endoscope System>

FIG. 1 is a block diagram illustrating a configuration example of anembodiment of an endoscope system according to the present technology.

The endoscope system in FIG. 1 is configured by an endoscope camera head11, a camera control unit (CCU) 12, an operation section 13, and adisplay 14.

This endoscope system is used in endoscopic surgery in which a region(surgery region) in a body being a surgical target is captured as anobserved portion and is displayed on the display 14, and the observedportion is treated while viewing the display 14.

In the endoscopic surgery, for example, as illustrated in FIG. 2, aninsertion portion 25 of the endoscope camera head 11 and two pairs offorceps 81 (81A and 81B) being surgical instruments are inserted intothe body of a patient. The endoscope camera head 11 emits light from atip end of the insertion portion 25, illuminates a surgery region 82 ofthe patient, and images a state of the two pairs of forceps 81 and thesurgery region 82.

Here, an endoscope will be described as an example, but the presenttechnology may be also applied to an apparatus other than a medicalapparatus such as an endoscope. For example, the present technology maybe applied to an apparatus of executing some types of processes on aremarkable region corresponding to a surgery region by an instructingtool, a predetermined device, or the like corresponding to the surgicalinstrument.

The endoscope camera head 11 includes an imaging section 21, a lightsource 22, and a focus lens 23, as illustrated in FIG. 1.

The imaging section 21 includes at least two imaging sensors 24 of afirst imaging sensor 24 a and a second imaging sensor 24 b. The imagingsensor 24 is configured by, for example, a charge coupled device (CCD)sensor, a complementary metal oxide semiconductor (CMOS) sensor, or thelike. The imaging sensor 24 images a subject and generates an imageobtained as a result. The imaging sensor 24 may employ a high resolutionsensor which has the number of pixels of about 4000×about 2000 being thenumber of pixels of (horizontal direction)×(vertical direction) and iscalled a 4K camera. The two imaging sensors 24 are disposed at apredetermined distance from each other in a traverse direction andgenerate images having view point directions different from each otherto output the images to the CCU 12.

In this embodiment, images obtained by the two imaging sensors 24performing imaging are referred to as surgery region images. In thisembodiment, the first imaging sensor 24 a is set to be disposed on aright side and the second imaging sensor 24 b is set to be disposed on aleft side, and the surgery region image generated by the first imagingsensor 24 a is referred to as an R image and the surgery region imagegenerated by the second imaging sensor 24 b is referred to as an Limage.

The light source 22 is configured by, for example, a halogen lamp, axenon lamp, a light emitting diode (LED) light source, and the like andthe light source 22 emits light for illuminating the surgery region.

The focus lens 23 is configured by one or a plurality of lenses, and isdriven by a focus control section 46 (will be described later) of theCCU 12 and forms an image on an imaging surface of the imaging sensor 24by using incident light (image light) from the subject.

The CCU 12 is an image processing apparatus for processing the surgeryregion image obtained by the imaging section 21 of the endoscope camerahead 11 performing imaging. The CCU 12 is configured by a depthinformation generation section 41, a forceps position detection section42, a remarkable point estimation section 43, an image superpositionsection 44, an operation control section 45, and a focus control section46.

An R image and an L image which are generated and output in the imagingsection 21 are supplied to the depth information generation section 41and the image superposition section 44 of the CCU 12. One (for example,L image) of the R image and the L image is also supplied to the focuscontrol section 46.

The depth information generation section 41 generates depth informationof the surgery region image from the supplied R image and L image. Morespecifically, the depth information generation section 41 calculates aposition of each pixel of the surgery region image in a depth directionby using the supplied R image and L image and a principle oftriangulation.

A calculation method of a depth position of each pixel in the surgeryregion image will be described by using the principle of triangulationwith reference to FIG. 3 and FIG. 4.

First, the first imaging sensor 24 a and the second imaging sensor 24 bare arranged in a row at a distance T in the traverse direction, asillustrated in FIG. 3, and each of the first imaging sensor 24 a and thesecond imaging sensor 24 b images an object P in the real world.

The positions of the first imaging sensor 24 a and the second imagingsensor 24 b in the vertical direction are the same as each other and thepositions in the horizontal direction are different from each other.Thus, the position of the object P in the R image obtained by the firstimaging sensor 24 a and the position of the object P in the L imageobtained by the second imaging sensor 24 b are different only in xcoordinates.

For example, the x coordinate of the object P shown in the R imageobtained by the first imaging sensor 24 a is set to x^(r) and the xcoordinate of the object P shown in the L image obtained by the secondimaging sensor 24 b is set to x^(l).

If the principle of triangulation is used, as illustrated in FIG. 4, thex coordinate of the object P in the R image being x^(r) corresponds to aposition on a straight line joining an optical center O_(r) of the firstimaging sensor 24 a and the object P. The x coordinate of the object Pin the L image being x^(l) corresponds to a position on a straight linejoining an optical center O_(l) of the second imaging sensor 24 b andthe object P.

Here, when a distance from the optical center O_(r) to an image plane ofthe R image or from the optical center O_(l) to an image plane of the Limage is set as f and a distance (depth) from the a line joining theoptical center O_(r) and the optical center O_(l) to the object P in thereal world is set as Z, parallax d is represented by d=(x^(l)=x^(r)).

A relationship of the Equation (1) is established between T, Z, d, andf.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{\frac{T - d}{Z - f} = \frac{T}{Z}} & (1)\end{matrix}$

Accordingly, the distance Z to the object P may be obtained by using thefollowing Equation (2) which is obtained by deforming the Equation (1).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack & \; \\{Z = \frac{fT}{x^{l} - x^{r}}} & (2)\end{matrix}$

The depth information generation section 41 in FIG. 1 calculates a depthZ of each pixel in the surgery region image by using the above-describedprinciple of the triangulation. The depth Z of each pixel calculated bythe depth information generation section 41 is supplied to the forcepsposition detection section 42 and the remarkable point estimationsection 43, as depth information.

The forceps position detection section 42 detects a position of animportant object such as the forceps 81 shown in the surgery regionimage using the depth information of the surgery region image suppliedfrom the depth information generation section 41. As described above,the two pairs of forceps 81 may be imaged as subjects in the surgeryregion image. However, a position of either of the forceps 81 may bedetected. The position of the distal end of the forceps 81 may also bedetected. The forceps 81 of which the position is to be detected may bedetermined in advance or the forceps of which the position is detectedmore easily than another in the surgery region image may be determined.In addition, the positions of the two pairs of forceps 81 may also bedetected.

Position detection of the forceps 81 performed by the forceps positiondetection section 42 will be described with reference to FIG. 5A to FIG.5D.

First, the forceps position detection section 42 generates a parallaximage from depth information of the surgery region image supplied fromthe depth information generation section 41. The parallax image refersto an image obtained by representing the depth Z of each pixel being thedepth information, in gray scale.

FIG. 5A illustrates an example of the parallax image and represents thatbrightness value in the parallax image becomes greater, correspondingdepth Z becomes less, and the subject in the surgery region imagebecomes closer to the front.

Then, the forceps position detection section 42 detects an edge which isa boundary between brightness values, from the generated parallax image.For example, pixels which have a difference between pixel values ofadjacent pixels is equal to or greater than a predetermined value in theparallax image are detected as an edge. Alternatively, the forcepsposition detection section 42 may detect be detected from one or more ofcolor difference information, brightness, and depth independent from orin concert with edge detection techniques.

An edge detection component is detected based on the brightness value.However, for example, the surgical field has red as a main component andthe forceps has a color such as silver, white, and black different fromred in general. Since the surgical field and the forceps have differentcolors as described above, edge detection based on color componentinformation may be also performed. That is, a configuration in which athree-dimensional position of the surgical instrument such as forceps isdetected based on information on a specific color in the parallax imagemay be made.

Here, a case of using the brightness value will be described as anexample. FIG. 5B illustrates an example of edges detected in theparallax image of FIG. 5A.

Then, the forceps position detection section 42 removes a curved edgeout of the detected edge and detects only a linear edge having apredetermined length or greater.

Since the forceps 81 has a bar shape, there is the linear edge having apredetermined length or greater, as the edge of the forceps 81. Thus,the forceps position detection section 42 only detects the linear edgehaving a predetermined length or greater out of the detected edge as theedge of the forceps 81.

The forceps position detection section 42 may determine whether or notthe detected linear edge is a straight line continuing from an outercircumference portion of the surgery region image in addition todetermining whether or not the detected linear edge has thepredetermined length or greater, when the edge of the forceps 81 isspecified. When the insertion portion 25 of the endoscope camera head 11and the forceps 81 have a position relationship as illustrated in FIG.2, the forceps 81 generally is captured to have a position of beingextended to a center portion from the outer circumference portion of thesurgery region image in the surgery region image. For this reason, it ispossible to further raise detection accuracy of the forceps 81 bydetermining whether or not the detected linear edge is a straight linecontinuing from the outer circumference portion of the surgery regionimage.

Then, the forceps position detection section 42 estimates a position ofthe forceps 81 in the three-dimensional space in the captured image,that is, a posture of the forceps 81, from the detected linear edge.

Specifically, the forceps position detection section 42 calculates aline segment (straight line) 101 corresponding to the forceps 81, fromthe detected linear edge, as illustrated in FIG. 5D. The line segment101 may be obtained by using an intermediate line between the detectedtwo linear edges, and the like.

The forceps position detection section 42 arbitrarily detects two points(x₁, y₁) and (x₂, y₂) on the calculated line segment 101 and acquiresdepth positions z₁ and z₂ at positions (x₁, y₁) and (x₂, y₂) of thedetected two points from the supplied depth information. Accordingly,the positions (x₁, y₁, z₁) and (x₂, y₂, z₂) of the forceps 81 in thethree-dimensional space are specified in the surgery region image. Thepositions may include, for example, the distal end of the forceps.

When two line segments corresponding to two pairs of forceps 81 aredetected in the surgery region image, either of the two line segmentsmay be selected by selecting one closer to the front than another.

Returning to FIG. 1, the forceps position detection section 42 suppliesthe positions (x₁, y₁, z₁) and (x₂, y₂, z₂) of the forceps 81 in thethree-dimensional space which are detected in the above-describedmanner, to the remarkable point estimation section 43.

The depth information of the surgery region image is supplied from thedepth information generation section 41 to the remarkable pointestimation section 43 and the coordinates (x₁, y₁, z₁) and (x₂, y₂, z₂)of the two points in the three-dimensional space which represent aposture of the forceps 81 are supplied from the forceps positiondetection section 42 to the remarkable point estimation section 43.

Alternatively to the forceps position detection section 42, element 42can also be an area position detection section. The area positiondetection section 42 detects an important area having, for example,certain tissues, body parts, bleeding or blood vessels, etc. Thedetection of the important area is based on color information,brightness and/or differences between different frames. For example, animportant area could be detected as an area having no bleeding in oneframe and then bleeding in subsequent frames.

The remarkable point estimation section 43 assumes that a remarkablepoint Q at the surgery region 82 is at a position obtained by extendingthe detected positions of the forceps 81 and estimates a position of theremarkable point Q of the surgery region 82 in the three-dimensionalspace, as illustrated in FIG. 6. The remarkable point Q at the surgeryregion 82 corresponds to an intersection point of an extension lineobtained by extending the detected posture of the forceps 81 and asurface of the surgery region 82. An estimated location coordinate ofthe remarkable point Q at the surgery region 82 in the three-dimensionalspace is supplied to the image superposition section 44.

The remarkable point estimation section 43 can also estimate theremarkable point Q from the determined important area. In particular,the remarkable point Q can be generated based on the position of theimportant area.

The image superposition section 44 generates a superposition image bysuperposing a predetermined mark (for example, a mark of x), a circle ora quadrangle as the region of interest having a predetermined size inwhich the remarkable point Q supplied from the remarkable pointestimation section 43 is set as the center on a position at which theremarkable point Q of the surgery region image supplied from the imagingsection 21 is set as the center and the image superposition section 44displays the generated superposition image on the display 14.

A configuration in which display mode information which is controlinformation for designating ON or OFF of the 3D display is supplied tothe image superposition section 44 from the operation control section 45may be also made. The image superposition section 44 supplies any one ofthe R image and the L image to the display 14 and causes the surgeryregion image to be displayed in the 2D display manner when OFF of the 3Ddisplay is designated through the display mode information.

On the other hand, when an instruction of ON of the 3D display isreceived through the display mode information, the image superpositionsection 44 supplies both of the R image and the L image to the display14 and causes the surgery region image to be displayed in a 3D manner.Here, the 3D display refers to an image display manner in which the Rimage and the L image are alternately displayed on the display 14, theright eye of a practitioner visually recognizes the R image, the lefteye of the practitioner visually recognizes the L image, and thus thepractitioner perceives the surgery region image three-dimensionally.

The operation control section 45 supplies various control signals tonecessary sections based on an operation signal supplied from theoperation section 13. For example, the operation control section 45supplies an instruction of focus matching to the focus control section46 in accordance with an instruction of matching a focus with an areaincluding the remarkable point Q generated in the operation section 13.

The focus control section 46 performs focus control by using a contrastmethod, based on the L image supplied from the imaging section 21.Specifically, the focus control section 46 drives the focus lens 23 ofthe endoscope camera head 11 and compares contrast of the L imagesupplied from the imaging section 21 to detect a focus position. Thefocus control section 46 may perform the focus control in which thelocation coordinate of the remarkable point Q is acquired from theremarkable point estimation section 43 and an area of a predeterminedrange having the remarkable point Q as the center is set to be a focuscontrol target area.

The operation section 13 includes at least a foot pedal 61. Theoperation section 13 receives an operation from a practitioner(operator) and supplies an operation signal corresponding to anoperation performed by the practitioner to the operation control section45. The practitioner may perform, for example, matching a focus with aposition of the mark indicating the remarkable point Q displayed on thedisplay 14, switching the 2D display and the 3D display of the surgeryregion image displayed on the display 14, setting zoom magnification ofthe endoscope, and the like by operating the operation section 13.

The display 14 is configured by, for example, a liquid crystal display(LCD) and the like and displays a surgery region image captured by theimaging section 21 of the endoscope camera head 11 based on an imagesignal supplied from the image superposition section 44. When thesuperposition mode is set to be ON, a surgery region image captured bythe imaging section 21 or a superposition image obtained by superposinga mark which has a predetermined shape and indicates a position of theremarkable point Q estimated by the remarkable point estimation section43 on the surgery region image is displayed on the display 14.

FIG. 7 illustrates an example of the superposition image displayed onthe display 14.

In a superposition image 100 of FIG. 7, an area of interest (region ofinterest) QA which is determined to be an area including the remarkablepoint Q and has a predetermined size is indicated on a surgery regionimage 110 supplied from the imaging section 21 by a quadrangular mark.

In the surgery region image 110, the two pairs of forceps 81A and 81Bare imaged and the remarkable point Q is estimated based on a positionof the forceps 81A on the left side. In the superposition image 100, theestimated remarkable point Q, the mark (quadrangle indicating the areaof interest in FIG. 7) for causing the practitioner to recognize theremarkable point Q, and a guide line 111 corresponding to an extensionline calculated through estimation of the remarkable point Q aredisplayed. The area of interest (region of interest) QA may beoverlapping on or adjacent to and/or distinct from the region includingthe forceps 81 or the important area.

In a configuration in which the guide line 111 is displayed, display ofthe guide line 111 allows a three-dimensional distance in the abdominalcavity to be recognized intuitively and may cause three-dimensionaldistance information to be provided for the practitioner in a plan view.

In FIG. 7, the quadrangle is illustrated as a mark or highlight forcausing the practitioner to recognize the remarkable point Q, but themark is not limited to the quadrangle. As the mark, other shapes such asa triangle may be applied. In addition, a mark of a shape such as an (x)mark may be displayed.

<Flow of Focusing Process>

Then, in the endoscope system of FIG. 1, a process when a remarkablepoint Q is detected and a focus is matched with a position of theremarkable point Q will be described with reference to a flowchart ofFIG. 8. Whether or not such a process (process of auto-focusing) ofdetecting a remarkable point Q and performing matching of a focus isexecuted may be set by a practitioner. A configuration in which aprocess of the flowchart illustrated in FIG. 8 is executed whenauto-focusing is set to be executed may be made.

In addition, a configuration in which setting whether or notauto-focusing is executed is performed by operating the foot pedal 61may be made.

A configuration in which a process based on the flow of the focusingprocess illustrated in FIG. 8 is started when a practitioner operates astart button (the foot pedal 61 is available), when an optical zoom andthe like is determined to be out of focus, and the like may be made. Thepresent technology may be also applied to a microscope system and have aconfiguration in which the process based on the flow of the focusingprocess illustrated in FIG. 8 is started using movement of an arm as atrigger in the microscope system.

FIG. 8 illustrates the flowchart of the focusing process. Power issupplied to each mechanism of the endoscope system in a state where thefocusing process of FIG. 8 is executed. The insertion portion 25 of theendoscope camera head 11 and the forceps 81 are inserted into the bodyof a patient and the light source 22 illuminates the surgery region 82of the patient.

First, in Step S1, the depth information generation section 41 generatesdepth information of a surgery region image from an R image and an Limage supplied from the imaging section 21 of the endoscope camera head11. More specifically, the depth information generation section 41calculates depth Z of the each location (pixel) in the surgery regionimage by using the Equation (2) which uses the principle oftriangulation described with reference to FIG. 4. Depth informationcalculated in the depth information generation section 41 is supplied tothe remarkable point estimation section 43 and the forceps positiondetection section 42.

A process of Step S1 is a process for determining three-dimensionalpositions of a surface of a subject imaged by the imaging section 21.

In Step S2, the forceps position detection section 42 executes a forcepsposition detecting process of detecting a position of the forceps 81 inthe surgery region image using the depth information of the surgeryregion image supplied from the depth information generation section 41.A process of Step S2 is a process for detecting a three-dimensionalposition of a bar shaped instrument such as forceps held by apractitioner. Step S2 can alternatively correspond to the area positiondetection section 42 executing an area position detecting process.

<Detailed Flow of Forceps Position Detecting Process>

FIG. 9 illustrates a detailed flowchart of the forceps positiondetecting process executed in Step S2.

In the forceps position detecting process, at first, in Step S21, theforceps position detection section 42 generates a parallax image fromthe depth information of the surgery region image supplied from thedepth information generation section 41.

In Step S22, the forceps position detection section 42 detects an edgewhich is a boundary between brightness values from the generatedparallax image.

In Step S23, the forceps position detection section 42 removes a curvededge out of the detected edge and detects only a linear edge having apredetermined length or greater.

In Step S24, the forceps position detection section 42 estimates aposition of the forceps 81 in the surgery region image in thethree-dimensional space from the detected linear edge. With this, asdescribed above with reference to FIG. 5D, coordinates (x₁, y₁, z₁) and(x₂, y₂, z₂) of two points indicating positions of the forceps 81 in thesurgery region image in the three-dimensional space are determined. Thepositions may be for example the distal end of the forceps.

The positions (x₁, y₁, z₁) and (x₂, y₂, z₂) of the forceps 81 in thesurgery region image, which are detected as described above, aresupplied to the remarkable point estimation section 43 and the processproceeds to Step S3 in FIG. 8.

In Step S3, the remarkable point estimation section 43 executes aremarkable point estimating process of assuming that a remarkable pointQ at the surgery region 82 is at a position obtained by extending thedetected positions of the forceps 81 and of detecting a position of theremarkable point Q of the surgery region 82 in the three-dimensionalspace.

<Detailed Flow of Remarkable Point Estimating Process>

The remarkable point estimating process executed in Step S3 of FIG. 8will be described in detail with reference to a flowchart of FIG. 10.

First, in Step S41, the remarkable point estimation section 43 obtainspositions (x₁, y₁, z₁) and (x₂, y₂, z₂) of the forceps 81 in the surgeryregion image in the three-dimensional space which are supplied from theforceps position detection section 42.

In Step S42, the remarkable point estimation section 43 calculates aslope a₁ of a line segment A joining the coordinates (x₁, y₁) and (x₂,y₂) of the two points of the forceps in an XY plane. The slope a₁ may becalculated by using the following equation.a ₁=(Y ₂ −Y ₁)/(X ₂ −X ₁)

In Step S43, the remarkable point estimation section 43 calculates aslope a₂ of a line segment B joining the coordinates (X₁, Z₁) and (X₂,Z₂) of the two points of the forceps in an XZ plane. The slope a₂ may becalculated by using the following equation.a ₂(Z ₂ −Z ₁)/(X ₂ −X ₁)

In Step S44, the remarkable point estimation section 43 determines acoordinate X₃ being an X coordinate value when the line segment A in theXY plane is extended by a predetermined length W in a center directionof the screen. The predetermined length W can be defined as 1/N (N is apositive integer) of the line segment A, for example.

In Step S45, the remarkable point estimation section 43 calculatesY₃=a₁*X₃ and calculates an extension point (X₃, Y₃) of the line segmentA in the XY plane. Here, “*” represents multiplication.

In Step S46, the remarkable point estimation section 43 calculates depthZ₃ of the extension point (X₃,Y₃) of the line segment A in the XZ planeby using Z₃=a₂*X₃. Here, the calculated depth Z₃ of the extension point(X₃, Y₃) of the line segment A in the XZ plane corresponds to a logicalvalue of the extension point (X₃, Y₃).

In Step S47, the remarkable point estimation section 43 acquires depthZ₄ of the extension point (X₃, Y₃) of the line segment A from the depthinformation supplied from the depth information generation section 41.Here, the acquired depth Z₄ of the extension point (X₃, Y₃) correspondsto a real value of the extension point (X₃, Y₃).

In Step S48, the remarkable point estimation section 43 determineswhether or not the depth Z₃ being the logical value of the extensionpoint (X₃, Y₃) is greater than the depth Z₄ being the real value of theextension point (X₃, Y₃).

A case where the extension point (X₃, Y₃) obtained by extending the linesegment A which corresponds to the forceps 81 by the predeterminedlength W in the center direction of the screen is not included in thesurgery region 82 means a case where the surgery region 82 is at aposition deeper than the extension point (X₃, Y₃). In this case, thedepth Z₄ being the real value of the extension point (X₃, Y₃) obtainedfrom the depth information is greater than the depth Z₃ being thelogical value.

On the other hand, when the surgery region 82 actually includes theextension point (X₃, Y₃), the real value (depth Z₄) of the extensionpoint (X₃, Y₃) obtained from the depth information becomes ahead of thelogical value (depth Z₃) of the extension point (X₃, Y₃). Thus, thedepth Z₄ being the real value of the extension point (X₃, Y₃) is lessthan the depth Z₃ being the logical value.

Accordingly, in Step S48, the remarkable point estimation section 43determines whether or not the logical value (depth Z₃) of the extensionpoint (X₃, Y₃) obtained by extending the line segment A by thepredetermined length W becomes ahead of the real value (depth Z₄) of theextension point (X₃, Y₃) obtained from the depth information.

In Step S48, when the depth Z₃ being the logical value of the extensionpoint (X₃, Y₃) is equal to or less than the depth Z₄ being the realvalue of the extension point (X₃,Y₃), that is, when the depth Z₃ beingthe logical value of the extension point (X₃, Y₃) is determined to beahead of the depth Z₄ being the real value of the extension point (X₃,Y₃), the process returns to Step S44.

In Step S44 to which the process returns, a coordinate X₃ obtained byextending the line segment A in the center direction of the screen to bedeeper than the current extension point (X₃, Y₃) by the predeterminedlength W in the XY plane is determined as a new coordinate X₃. Steps S45to S48 which are described above are executed again on the determinednew coordinate X₃.

On the other hand, in Step S48, when the depth Z₃ being the logicalvalue of the extension point (X₃, Y₃) is greater than the depth Z₄ beingthe real value, that is, when the depth Z₄ being the real value of theextension point (X₃, Y₃) is determined to be ahead of the depth Z₃ beingthe logical value, the process proceeds to Step S49.

In Step S49, the remarkable point estimation section 43 determines anextension point (X₃, Y₃, Z₄) which has the depth Z₄ being the real valueas a Z coordinate value to be the remarkable point Q.

In this manner, Step S3 of FIG. 8 is completed and the process proceedsto Step S4.

In Step S4, the estimated (detected) remarkable point Q is displayed ona screen. For example, a predetermined mark is displayed at the positionof the remarkable point Q, as described with reference to FIG. 7.

In Step S5, it is determined whether or not an instruction of setting afocal point is generated. The practitioner operates the foot pedal 61 ofthe operation section 13 when the practitioner wishes to control theremarkable point Q displayed on the display 14 to be a focal point. Whensuch this operation is performed, it is determined that the instructionof controlling the estimated remarkable point to be focal point isgenerated.

Here, it is described that operating the foot pedal 61 causes aninstruction of setting a focal point. However, for example, aconfiguration will be made in which an instruction of setting a focalpoint is generated using other methods such as a method in which aswitch (not illustrated) attached to the forceps is operated and amethod in which a word preset through voice recognition is spoken.

In Step S5, when it is determined that the instruction of setting afocal point is not generated, the process returns to Step S1 and thesubsequent processes are repeated.

On the other hand, in Step S5, when it is determined that theinstruction of setting a focal point is generated, the process proceedsto Step S6 and the focus control section 46 performs the focus control.The focus control section 46 performs the focus control in such a mannerthat the focus control section 46 obtains information on a coordinateposition and the like of the remarkable point estimated by theremarkable point estimation section 43 and sets a focus to match withthe position of the remarkable point Q with the obtained information by,for example, the above-described contrast method.

Meanwhile, when it is determined that the instruction of setting a focalpoint is not generated in Step S5, as illustrated in FIG. 8, the processreturns to Step S1, depth information is generated again, and thesubsequent processes are repeated in a case where three-dimensionalpositions of the surface of the imaged subject may be changed due tomovement of the endoscope camera head 11.

The present technology may be applied to a microscope system other thanthe above-described endoscope system, similarly. In this case, amicroscope camera head may be provided instead of the endoscope camerahead 11 in FIG. 1 and the CCU 12 may execute a superposition displayingprocess of the surgery region image imaged by the microscope camerahead.

In the microscope system, a configuration in which when the instructionof setting a focal point is not generated in Step S5 illustrated in FIG.8, the process returns to Step S2 and the subsequent processes arerepeated may be made in a case where the microscope camera head and thesubject are fixed and thus it is unlikely that the three-dimensionalpositions of the surface of the subject imaged by the imaging section 21are changed. In this case, the three-dimensional positions (depthinformation) of the surface of the imaged subject are obtained once atfirst and then the processes are executed by repeatedly using theobtained depth information.

For example, depth information may be also generated in Step S1 for eachpredetermined period and the generated depth information may be updatedin a configuration in which the process returns to Step S2 and thesubsequent processes are repeated when it is determined that theinstruction of setting a focal point is not generated in Step S5.

In the above-described forceps position detecting process, an example inwhich the forceps 81 is detected by generating a parallax image from thedepth information of the surgery region image and detecting a linearedge will be described. However, other detecting methods may beemployed.

For example, marks may be marked on two predetermined locations of theforceps 81 and the two marks marked on the forceps 81 may be detected aspositions (x₁,y₁,z₁) and (x₂,y₂,z₂) of the forceps 81 in the surgeryregion image. Characteristics such as a shape and a color of the forceps81, and a shape and a color of the mark marked on the forceps 81 ofvarious forcipes 81 used in performing surgery are stored in advance ina memory of the forceps position detection section 42 as a database andthe mark may be detected based on information on the designated forceps81.

In this manner, since a direction pointed by the forceps is a directionon which the practitioner focuses and a focus is controlled to matchwith a portion of the subject image positioned in the direction, thepractitioner may make a focus match with a desired portion withoutseparating the practitioner from the instrument such as the forceps.Accordingly, it is possible to provide an endoscope system or amicroscope system having good convenience and improved operability.

<The Other Flow of Focusing Process>

Other process according to the focus control of the endoscope systemwill be described with reference to a flowchart illustrated in FIG. 11.

In Step S101, the forceps position detection section 42 executes theforceps position detecting process of detecting a position of theforceps 81 in the surgery region image. The process in Step S101corresponds to a process for detecting a three-dimensional position of abar shaped instrument such as a forceps held by the practitioner. Theforceps position detecting process in Step S101 may be executedsimilarly to the process of Step S2 illustrated in FIG. 8.

In Step S102, the depth information generation section 41 generatesdepth information of the surgery region image from an L image and an Rimage supplied from the imaging section 21 of the endoscope camera head11. A process of Step S102 corresponds to a process for determiningthree-dimensional positions of the surface of the subject imaged by theimaging section 21. The forceps position detecting process in Step S102may be executed similarly to the process of Step S1 illustrated in FIG.8.

A process based on a flowchart illustrated in FIG. 11 is obtained byreversing an order of generating depth information and then detecting aposition of the forceps in the process based on the flowchartillustrated in FIG. 8 and is executed in order of detecting the positionof the forceps and then generating the depth information.

In detecting the position of the forceps, a portion which is on anextension of the tip end (distal end) portion of the forceps andoverlapped with the subject image may be estimated. The process forgenerating the depth information is executed on the estimated portion ofthe subject image. That is, an area for generating depth information isextracted and the depth information in the extracted area is generatedin the process of the flowchart illustrated in FIG. 11.

In this manner, it is possible to reduce processing capacity orprocessing time necessary for generating depth information by extractingan area for generating depth information.

Processes of Step S103 to Step S106 are executed similarly to theprocesses of Step S3 to Step S6 in the flowchart of FIG. 8. Thus,descriptions thereof are omitted.

As described above, according to the present technology, a focus may bematched with an image at a portion pointed by the forceps and thus it ispossible to match a focus with a desired portion without separating aninstrument such as the forceps held with the hand by the an operatorfrom the hand. Accordingly, it is possible to improve operability of theendoscope system.

<Regarding Recording Medium>

An image process executed by the CCU 12 may be executed with hardware orsoftware. When a series of processes are executed with software, aprogram constituting the software is installed on a computer. Here, thecomputer includes a computer obtained by combining dedicated hardware, ageneral personal computer allowed to perform various functions byinstalling various programs, and the like.

FIG. 12 is a block diagram illustrating a configuration example ofhardware of a computer in which the CCU 12 executes an image process byusing a program.

In the computer, a central processing unit (CPU) 201, a read only memory(ROM) 202, and a random access memory (RAM) 203 are connected to eachother through a bus 204.

An input and output interface 205 is connected to the bus 204. An inputsection 206, an output section 207, a storage section 208, acommunication section 209, and a drive 210 are connected to the inputand output interface 205.

The input section 206 is configured by a keyboard, a mouse, amicrophone, and the like. The output section 207 is configured by adisplay, a speaker, and the like. The storage section 208 is configuredby a hard disk, a non-volatile memory, and the like. The communicationsection 209 is configured by a network interface and the like. The drive210 drives a removable recording medium 211 such as a magnetic disk, anoptical disc, a magneto-optical disk, and a semiconductor.

In the computer configured as described above, the CPU 201 executes theabove-described series of processes by loading a program stored in thestorage section 208 on the RAM 203 through the input and outputinterface 205 and the bus 204 and executing the loaded program, forexample.

In the computer, the program may be installed on the storage section 208through the input and output interface 205 by mounting the removablerecording medium 211 on the drive 210. The program may be received inthe communication section 209 through a wired or wireless transmissionmedium such as a local area network, the Internet, and satellite databroadcasting and may be installed on the storage section 208. Inaddition, the program may be installed on the ROM 202 or the storagesection 208 in advance.

In this specification, the steps illustrated in the flowcharts may beexecuted in time series in the illustrated order. The steps may beexecuted not necessarily in time series, but in parallel or at anecessary timing, for example, when calling is performed, and the like.

In this specification, the system means a set of multiple constituents(apparatus, module (component), and the like) and it is not necessarythat all of the constituents are in the same housing. Accordingly, thesystem includes a plurality of apparatuses which are stored in separatehousings and connected to each other through a network and one apparatusin which a plurality of modules are stored in one housing.

The embodiment of the present technology is not limited to theabove-described embodiment and may be changed variously withoutdeparting the gist of the present technology.

For example, an embodiment obtained by combining some or all of theabove-described embodiments may be employed.

For example, the present technology may have a configuration of cloudcomputing in which one function is distributed by a plurality ofapparatuses through a network and the plurality of apparatuses togetherprocess the function.

Each step illustrated in the above-described flowcharts may be executedin one apparatus or may be distributed and executed in a plurality ofapparatuses.

Furthermore, when one step includes a plurality of processes, theplurality of processes included in the one step may be executed in oneapparatus or may be distributed and executed in a plurality ofapparatuses.

The effects described in this specification are only examples and arenot limited thereto. There may be effects other than the effectsdescribed in this specification.

The present technology may have the following configurations.

(1)

A medical image processing apparatus, comprising:

a controller including circuitry configured to

determine a position of a distal end of an important object within amedical image,

estimate a region of interest within the medical image adjacent to aregion including the important object based on the position of thedistal end of the important object, and

control display of the region of interest.

(2)

The medical image processing apparatus according to (1), wherein themedical image includes a surgical image of a body.

(3)

The medical image processing apparatus according to (1) to (2), whereinthe region of interest within the medical image is distinct from theregion including the important object.

(4)

The medical image processing apparatus according to (1) to (3), whereinthe controller including the circuitry is configured to determine theposition of the distal end of the important object within the medicalimage based on three dimensional position information of the importantobject.

(5)

The medical image processing apparatus according to (1) to (4), whereinthe controller including the circuitry is configured to control displayof the region of interest by displaying one or more of: a zoomed imagecorresponding to the region of interest, the medical image having focuson the region of interest, and the medical image in which the region ofinterest is highlighted.

(6)

The medical image processing apparatus according to (1) to (5), whereinthe controller including the circuitry is configured to estimate theregion of interest based on a three-dimensional posture of the importantobject.

(7)

The medical image processing apparatus according to (1) to (6), whereinthe controller including the circuitry is configured to estimate theregion of interest by detecting an estimated position of a cross pointbetween an extension from the distal end of the important object and asurface of a body.

(8)

The medical image processing apparatus according to (1) to (7), whereinthe controller including the circuitry is configured to determine theposition of the distal end of the important object within the medicalimage by detecting the position of the distal end of the importantobject using one or more of color difference information, edge detectiontechniques, brightness, and depth.

(9)

A method for processing a medical image by a medical image processingapparatus including a controller including circuitry, the methodcomprising:

determining, using the circuitry, a position of a distal end of animportant object within the medical image;

estimating, using the circuitry, a region of interest within the medicalimage adjacent to a region including the important object based on theposition of the distal end of the important object; and

controlling, using the circuitry, display of the region of interest.

(10)

A non-transitory computer readable medium having stored thereon aprogram that when executed by a computer causes the computer toimplement a method for processing a medical image by a medical imageapparatus including a controller including circuitry, the methodcomprising:

determining, using the circuitry, a position of a distal end of animportant object within the medical image;

estimating, using the circuitry, a region of interest within the medicalimage adjacent to a region including the important object based on theposition of the distal end of the important object; and

controlling, using the circuitry, display of the region of interest.

(11)

A medical image processing system, comprising:

a medical imaging device that obtains a medical image;

a display device having a display area; and

a controller including circuitry configured to

determine a position of a distal end of an important object within themedical image obtained by the medical imaging device,

estimate a region of interest within the medical image adjacent to aregion including the important object based on the position of thedistal end of the important object, and control display of the region ofinterest in the display area of the display device.

(12)

The medical image processing system according to (11), wherein themedical imaging device generates both a left and a right medical imagecorresponding to the medical image using three dimensional imaging.

(13)

The medical image processing system according to (11) to (12), whereinthe controller including the circuitry is configured to determine theposition of the distal end of the important object within the medicalimage by detecting the position of the distal end of the importantobject using a depth image generated from the left and right medicalimages.

(14)

The medical image processing system according to (11) to (13), whereinthe important object is a surgical instrument and the medical imagingdevice is an endoscope or a surgical microscope.

(15)

A medical image processing apparatus, comprising:

a controller including circuitry configured to

determine a position of an important area within a medical image,

estimate a region of interest within the medical image adjacent to aregion including the important area based on the position of theimportant area, and

control display of the region of interest.

(16)

The medical image processing apparatus according to (15), wherein theregion of interest within the medical image is distinct from the regionincluding the important area.

(17)

An image processing apparatus including: a generation section configuredto generate depth information of an image from the image obtained byimaging a surgical field which includes a region of a surgical target;and a position detection section configured to detect athree-dimensional position of a surgical instrument by using thegenerated depth information of the image.

(18)

The apparatus according to (17), further including: a remarkable pointestimation section configured to estimate a remarkable point for apractitioner operating the surgical instrument, based on the detectedthree-dimensional position of the surgical instrument and the depthinformation.

(19)

The apparatus according to (18), further including: a focus controlsection configured to output a focus control signal such that a focuscoincides with the remarkable point estimated by the remarkable pointestimation section.

(20)

The apparatus according to (18), in which the remarkable pointestimation section estimates an intersection point to be the remarkablepoint, the intersection point of the surgery region and an extensionline obtained by extending a line segment corresponding to thethree-dimensional position of the surgical instrument.

(21)

The apparatus according to (18), in which a predetermined mark forindicating the remarkable point is superposed on the image obtained bythe imaging section.

(22)

The apparatus according to (19), in which the focus control sectioncontrols the focus when an instruction from the practitioner isreceived, and the instruction from the practitioner is generated by anoperation of a foot pedal or a button included in the surgicalinstrument or utterance of a predetermined word.

(23)

The apparatus according to any one of (17) to (22), in which thegeneration section generates the depth information from a predeterminedarea of the image positioned on extension of a tip end portion of thesurgical instrument which is detected by the position detection section.

(24)

The apparatus according to (21), in which an extension line is alsosuperposed and displayed on the image, the extension line being obtainedby extending a line segment corresponding to the three-dimensionalposition of the surgical instrument.

(25)

The apparatus according to any one of (17) to (24), in which theposition detection section detects the three-dimensional position of thesurgical instrument based on information on a linear edge in the image.

(26)

The apparatus according to any one of (17) to (25), in which theposition detection section detects the three-dimensional position of thesurgical instrument based on information on a specific color in aparallax image.

(27)

The apparatus according to any one of (17) to (26), in which theposition detection section detects the three-dimensional position of thesurgical instrument by detecting a mark marked on the surgicalinstrument from the image.

(28)

An image processing method including: causing an image processingapparatus to generate depth information of an image from the imageobtained by imaging a surgical field which includes a region of asurgical target and to detect a three-dimensional position of thesurgical instrument by using the generated depth information of theimage.

(29)

A program of causing a computer to execute a process including:generating depth information of an image from the image obtained byimaging a surgical field which includes a region of a surgical target;and detecting a three-dimensional position of the surgical instrument byusing the generated depth information of the image.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

REFERENCE SIGNS LIST

-   -   11 Endoscope Camera head    -   12 CCU    -   13 Operation section    -   14 Display    -   21 Imaging section    -   24 a First imaging sensor    -   24 b Second imaging sensor    -   41 Depth information generation section    -   42 Forceps/area position detection section    -   43 Remarkable point estimation section    -   44 Image superposition section    -   45 Operation control section    -   61 Foot pedal    -   Q Remarkable point    -   QA Area of interest    -   111 Guide line    -   QB Zoom image    -   201 CPU    -   202 ROM    -   203 RAM    -   206 Input section    -   207 Output section    -   208 Storage section    -   209 Communication section    -   210 Drive

The invention claimed is:
 1. A medical image processing apparatus,comprising: a controller including circuitry configured to: generatedepth information of each location in a medical image from a right (R)image and a left (L) image supplied from an imaging section of themedical image processing apparatus using triangulation, determine aposition of a distal end of an important object within the medical imageby detecting a linear edge having a predetermined length or greater froma parallax image generated by the circuitry, wherein the linear edge isdetected by detecting a difference between the important object and asurgical field within the medical image, estimate a region of interestwithin the medical image adjacent to a region including the importantobject based on the position of the distal end of the important object,and control display of the region of interest, wherein the controllerincluding the circuitry is further configured to estimate the region ofinterest by detecting an estimated position of a cross point between anextension line from the distal end of the important object and a surfaceof a body, the extension line being in parallel with a longitudinaldirection of the important object, wherein the position of the crosspoint is estimated by comparing the depth of each point along theextension line with the depth information of a corresponding location inthe medical image until the depth information of a correspondinglocation in the medical image is less than the depth of a point alongthe extension line.
 2. The medical image processing apparatus accordingto claim 1, wherein the medical image includes a surgical image of thebody.
 3. The medical image processing apparatus according to claim 1,wherein the region of interest within the medical image is distinct fromthe region including the important object.
 4. The medical imageprocessing apparatus according to claim 1, wherein the controllerincluding the circuitry is configured to determine the position of thedistal end of the important object within the medical image based onthree dimensional position information of the important object.
 5. Themedical image processing apparatus according to claim 1, wherein thecontroller including the circuitry is configured to control display ofthe region of interest by displaying one or more of: a zoomed imagecorresponding to the region of interest, the medical image having focuson the region of interest, and the medical image in which the region ofinterest is highlighted.
 6. The medical image processing apparatusaccording to claim 1, wherein the controller including the circuitry isconfigured to estimate the region of interest based on athree-dimensional posture of the important object.
 7. The medical imageprocessing apparatus according to claim 1, wherein the controllerincluding the circuitry is configured to determine the position of thedistal end of the important object within the medical image by detectingthe position of the distal end of the important object using one or moreof color difference information, edge detection techniques, brightness,and depth.
 8. A method for processing a medical image by a medical imageprocessing apparatus including a controller including circuitry, themethod comprising: generating, using the circuitry, depth information ofeach location in the medical image from a right (R) image and a left (L)image supplied from an imaging section of the medical image processingapparatus using triangulation, determining, using the circuitry, aposition of a distal end of an important object within the medical imageby detecting a linear edge haying a predetermined length or greater froma parallax image generated by the circuitry, wherein the linear edge isdetected by detecting a difference between the important object and asurgical field within the medical image; estimating, using thecircuitry, a region of interest within the medical image adjacent to aregion including the important object based on the position of thedistal end of the important object, wherein estimating the region ofinterest is performed by detecting an estimated position of a crosspoint between an extension line from the distal end of the importantobject and a surface of a body, the extension line being in parallelwith a longitudinal direction of the important object, wherein theposition of the cross point is estimated by comparing the depth of eachpoint along the extension line with the depth information of acorresponding location in the medical image until the depth informationof a corresponding location in the medical image is less than the depthof a point along the extension line; and controlling, using thecircuitry, display of the region of interest.
 9. A non-transitorycomputer readable medium having stored thereon a program that whenexecuted by a computer causes the computer to implement a method forprocessing a medical image by a medical image processing apparatusincluding a controller including circuitry, the method comprising:generating, using the circuitry depth information of each location inthe medical image from a right (R) image and a left (L) image suppliedfrom an imaging, section of the medical image processing apparatus usingtriangulation, determining, using the circuitry, a position of a distalend of an important object within the medical image by detecting alinear edge having a predetermined length or greater from a parallaximage generated by the circuitry, wherein the linear edge is detected bydetecting a difference between the important object and a surgical fieldwithin the medical image; estimating, using the circuitry, a region ofinterest within the medical image adjacent to a region including theimportant object based on the position of the distal end of theimportant object, wherein estimating the region of interest is performedby detecting an estimated position of a cross point between an extensionline from the distal end of the important object and a surface of abody, the extension line being in parallel with a longitudinal directionof the important object, wherein the position of the cross point isestimated by comparing the depth of each point along the extension linewith the depth information of a corresponding location in the medicalimage until the depth information of a corresponding location in themedical image is less than the depth of a point along the extensionline; and controlling, using the circuitry, display of the region ofinterest.
 10. A medical image processing system, comprising: a medicalimaging device that obtains a medical image; a display device having adisplay area; and a controller including circuitry configured togenerate depth information of each location in the medical image from aright (R) image and a left (L) image supplied from an imaging section ofthe medical imaging device using triangulation, determine a position ofa distal end of an important object within the medical image obtained bythe medical imaging device by detecting a linear edge having apredetermined length or greater from a parallax image generated by thecircuitry, wherein the linear edge is detected by detecting a differencebetween the important object and a surgical field within the medicalimage, estimate a region of interest within the medical image adjacentto a region including the important object based on the position of thedistal end of the important object, and control display of the region ofinterest in the display area of the display device, wherein thecontroller including the circuitry is further configured to estimate theregion of interest by detecting an estimated position of a cross pointbetween an extension line from the distal end of the important objectand a surface of a body, the extension line being in parallel with alongitudinal direction of the important object, wherein the position ofthe cross point is estimated by comparing the depth of each point alongthe extension line with the depth information of a correspondinglocation in the medical image until the depth information of acorresponding location in the medical image is less than the depth of apoint along the extension line.
 11. The medical image processing systemaccording to claim 10, wherein the medical imaging device generates botha left and a right medical images corresponding to the medical imageusing a three dimensional imaging.
 12. The medical image processingsystem according to claim 11, wherein the controller including thecircuitry is configured to determine the position of the distal end ofthe important object within the medical image by detecting the positionof the distal end of the important object using a depth image generatedfrom the left and right medical images.
 13. The medical image processingsystem according to claim 10, wherein the important object is a surgicalinstrument and the medical imaging device is an endoscope or a surgicalmicroscope.
 14. A medical image processing apparatus, comprising: acontroller including circuitry configured to: generate depth informationof each location a medical image from a right (R) image and a left (L)image supplied from an ging section of the medical image processingapparatus using triangulation, determine a position of an important areawithin the medical image by detecting differences between the importantarea and a surgical field within the medical image, estimate a region ofinterest within the medical image adjacent to a region including theimportant area based on the position of the important area, and controldisplay of the region of interest, wherein the controller including thecircuitry is further configured to estimate the region of interest bydetecting an estimated position of a cross point between an extensionline from the distal end of the important area and a surface of a body,the extension line being in parallel with a longitudinal direction ofthe important area, wherein the position of the cross point is estimatedby comparing the depth of each point along the extension line with thedepth information of a corresponding location in the medical image untilthe depth information of a corresponding location in the medical imageis less than the depth of a point along the extension line.
 15. Themedical image processing apparatus according to claim 14, wherein theregion of interest within the medical image is distinct from the regionincluding the important area.